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A Method and System for Electronic Mail 

FIELD OF THE INVENTION 
The present invention relates to management of 
electronic mail. 

BACKGROUND OF THE INVENTION 
Electronic mail, or e-mail, is fast becoming a dominant 
mode of communication. In its primitive form, e-mail is managed 
by two kinds of computer software: namely, e-mail server software 
(referred to simply as "e-mail servers") and e-mail client soft- 
ware (referred to simply as "e-mail clients") . E-mail servers 
reside on centrally accessed computers (referred to as "server 
computers") , and manage incoming and outgoing e-mail for remote 
users who are authorized to use such server computers. E-mail 
servers maintain separate mail accounts for each authorized user, 
into which incoming mail is stored. E-mail clients reside on 
remote computers (referred to as "client computers") and manage 
incoming and outgoing mail for designated users. E-mail clients 
are responsible for downloading arriving mail items from the 
server computer at which they are located. E-mail clients are 
also responsible for transmitting outgoing mail from a remote 

i 

client computer to a server computer. Examples of popular e-mail 
clients for personal computers are EUDORA R , INTERNET EXPLORER R 
and NETSCAPE R . 

Standard e-mail uses a protocol named Simple Mail 
Transport Protocol (SMTP) for sending outgoing messages, and a 
protocol named Post Office Protocol (POP) for receiving incoming 
messages. Other techniques and protocols for sending and receiving 
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e-mail may also be used, as appropriate. For example web based e- 
mail, using the HTTP protocol may be employed. 

For use over the Internet, Multipart Internet Mail 
Extension (MIME) extends the format of Internet mail to allow 
non-US-ASCII textual messages, non-textual messages, multipart 
message bodies and non-US-ASCII information in message headers. 
Attachments to e-mail messages can be included within a multipart 
e-mail. One of the drawbacks with e-mail, however, is the need 
for a user to be connected to the Internet in order to receive or 
send his e-mail. 

Although Internet access is available in many homes and 
offices, and Internet stations are available in many public 
areas, being "un-wired" is still a concern for mobile users who 
are neither at home nor in their office. When on the road, a user 
often finds himself unable to access his e-mail, for a variety of 
reasons . 



2 



SUMMARY OF THE INVENTION 
There is thus provided in accordance with a preferred 
embodiment of the present invention a voice communication system 
including a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections, a 
computer network having a multiplicity of nodes and enabling e- 
mail communication between the nodes, a multiplicity of voice 
response computers, each voice response computer being connected 
to a node of the computer network and being actuable by an input 
received from one of the multiplicity of telephones via the 
telephone network for communicating voice received via the one of 
the multiplicity of telephones via e-mail over the computer 
network . 

There is also provided in accordance with another 
preferred embodiment of the present invention a voice 
communication system including a telephone network including a 
multiplicity of telephones interconnected by telephone network 
interconnections, a computer network having a multiplicity of 
nodes and enabling e-mail communication between the nodes, a 
multiplicity of voice response computers, each voice response 
computer being connected to a node of the computer network and 
being actuable by an input received from one of the multiplicity 
of voice response computers via the computer network for 
receiving voice communicated via e-mail over the computer network 
and providing a voice output to a telephone via the telephone 
network. 

There is also provided in accordance with yet another 
preferred embodiment of the present invention a voice 
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communication system including a telephone network including a 
network including a multiplicity of telephones interconnected by 
telephone network interconnections ; a computer network having a 
multiplicity of nodes and enabling e-mail communication between 
the nodes, a multiplicity of voice response computers, each voice 
response computer being connected to a node of the computer 
network and being actuable by an input received from one of the 
multiplicity of telephones via the telephone network for commu- 
nicating voice received via the one of the multiplicity of 
telephones via e-mail over the computer network, each voice 
response computer also being actuable by an input received from 
one of the multiplicity of voice response computers via the 
computer network for receiving voice communicated via e-mail 
over the computer network and providing a voice output to a 
telephone via the telephone network. 

There is further provided in accordance with another 
preferred embodiment of the present invention a voice 
communication system including a telephone network including a 
multiplicity of telephones interconnected by telephone network 
interconnections; a computer network having a multiplicity of 
nodes and enabling e-mail communication between the nodes, a 
multiplicity of voice response computers, each voice response 
computer being connected to a node of the computer network and 
being actuable by an input received from one of the multiplicity 
of telephones via the telephone network for communicating voice 
received via the one of the multiplicity of telephones via a non- 
streaming Internet protocol over the computer network. 
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There is also provided in accordance with another 
preferred embodiment of the present invention a voice 
communication system including a telephone network including a 
telephone network including a multiplicity of telephones 
interconnected by telephone network interconnections, a computer 
network having a multiplicity of nodes and enabling non-streaming 
Internet protocol communication between the nodes, a multiplicity 
of voice response computers, each voice response computer being 
connected to a node of the computer network and being actuable by 
an input received from one of the multiplicity of voice response 
computers via the computer network for receiving voice 
communicated via e-mail over the computer network and providing 
a voice output to a telephone via the telephone network. 

There is also provided in accordance with another 
preferred embodiment of the present invention a voice 
communication system including a telephone network including a 
telephone network including a multiplicity of telephones 
interconnected by telephone network interconnections, a computer 
network having a multiplicity of nodes and enabling e-mail 
communication between the nodes, a multiplicity of voice response 
computers, each voice response computer being connected to a node 
of the computer network and being actuable by an input received 
from one of the multiplicity of telephones via the telephone 
network for communicating voice received via the one of the 
multiplicity of telephones via a non-streaming Internet protocol 
over the computer network, each voice response computer also 
being actuable by an input received from one of the multiplicity 



of voice response computers via the computer network for 
receiving voice communicated via a non-streaming Internet 
protocol over the computer network and providing a voice output 
to a telephone via the telephone network. 

Further in accordance with a preferred embodiment of 
the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via the 
computer network. 

Still further in accordance with a preferred embodiment 
of the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via said 
computer network indicating that a user is communicating using a 
user's telephone via the telephone network with a user's voice 
response computer . 

Additionally in accordance with a preferred embodiment 
of the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via the 
computer network indicating that a user has communicated voice 
via the telephone network and the computer network using a user's 
telephone and a user's voice response computer. 

Preferably the voice response computer is operative to 
convert DTMF to a buddy communication protocol. 

There is also provided in accordance with yet another 
preferred embodiment of the present invention a communication 
system including a cellular telephone network including a 
multiplicity of telephones interconnected by telephone network 
interconnections, a computer network having a multiplicity of 
nodes and enabling e-mail communication between the nodes, a 



multiplicity of computers, each computer being connected to a 
node of the computer network and being actuable by an input 
received from one of the multiplicity of telephones via the 
telephone network for communicating messages received via the one 
of the multiplicity of telephones via HDML over the computer 
network . 

There is further provided in accordance with yet 
another preferred embodiment of the present invention a 
communication system comprising: a cellular telephone network 
including a multiplicity of telephones interconnected by 
telephone network interconnections, a computer network having a 
multiplicity of nodes and enabling e-mail communication between 
the nodes, a multiplicity of computers, each computer being 
connected to a node of the computer network and being actuable by 
an input received from one of the multiplicity of voice response 
computers via the computer network for receiving messages 
communicated via HDML over the computer network and providing a 
HDML outputhfeoea telephone via the telephone network, 
is also provided in accordance with another preferred embodiment 
of the present invention a communication system including a 
cellular telephone network including a multiplicity of telephones 
interconnected by telephone network interconnections, a computer 
network having a multiplicity of nodes and enabling e-mail 
communication between the nodes, a multiplicity of computers, 
each computer being connected to a node of the computer network 
and being actuable by an input received from one of the 
multiplicity of telephones via the telephone network for 



communicating messages received via the one of the multiplicity 
of telephones via HDML over the computer network, each computer 
also being actuable by an input received from one of the 
multiplicity of computers via the computer network for receiving 
messages communicated via e-mail over the computer network and 
providing a HDML output to a telephone via the telephone network. 

Further in accordance with a preferred embodiment of 
the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via the 
computer network. 

Still further in accordance with a preferred embodiment 
of the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via the 
computer network indicating that a user is communicating using a 
user's telephone via the telephone network with a user's voice 
response computer. 

Additionally in accordance with a preferred embodiment 
of the present invention and also providing buddy functionality 
whereby communications are sent to user-selected buddies via said 
computer network indicating that a user has communicated voice 
via said telephone network and said computer network using a 
user's telephone and a user's voice response computer. 

Preferably the voice response computer is operative to 
convert DTMF to a buddy communication protocol. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention will be more fully understood and 
appreciated from the following detailed description, taken in 
conjunction with the drawings in which: 

Fig, 1 is a simplified illustration of a voice response 
computer used for processing voice electronic mail in a preferred 
embodiment of the present invention; 

Figs. 2A and 2B are simplified illustrations of an e- 
mail management system that posts a recorded message as an audio 
file and posts a reply message as an audio file on an Internet 
web site, in accordance with a preferred embodiment of the 
present invention, Fig. 2A representing the functionality of 
posting a recorded message as an audio file and Fig. 2B repre- 
senting the functionality of posting a reply message as an audio 
file; 

Fig. 3 is a simplified illustration of an e-mail man- 
agement system that sends an audio file attachment to a reply e- 
mail, in accordance with a preferred embodiment of the present 
invention; 

Fig. 4 is a simplified illustration of how a user can 
route his e-mail to a pager and/or a fax machine, in accordance 
with a preferred embodiment of the present invention; 

Fig. 5 is a simplified illustration of how a user can 
send a reply e-mail to a sender's pager and/or a sender's fax 
machine, in accordance with a preferred embodiment of the present 
invention ; 

Fig. 6 is a simplified illustration of two users dial- 
ing in to a voice response computer operative in accordance with 



a preferred embodiment of the present invention; 

Fig. 7 is a simplified illustration of a queue for 
text-to-speech processing tasks in a preferred embodiment of the 
present invention ; 

Fig. 8 is a simplified illustration of initiation of a 
voice e-mail message from a telephone; 

Fig. 9 is a simplified illustration of initiation of a 
voice e-mail message using both a web server and a telephone; 

Fig. 10 is a simplified illustration of initiation of a 
voice e-mail message using a telephone which has a web interface; 
and 

Fig. 11 is a simplified illustration of initiation of 
an e-mail message using a telephone. 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 
The present specification describes a method and system 
for retrieving e-mail by dialing in from a telephone to a voice 
response unit. The e-mail is converted to speech and read to the 
user over the phone. Various selections can be made by pushing 
the buttons of the telephone, and the user can effectively scroll 
through his e-mail and choose the ones to listen to. 

The present specification also describes a method and 
system for replying to e-mail through a telephone. The reply 
takes one or both of two forms: (1) a text e-mail reply, generat- 
ed from the speech reply by a speech-to-text converter, and (2) 
an audio file that contains a recording of the speech reply 
therewithin. In one preferred embodiment of the present inven- 
tion, the audio file is sent back to the original sender as a 
MIME attachment within a reply e-mail, in a standard audio format 
such as a WAV file. 

When the original sender receives the reply e-mail, he 
can download the WAV file to his local computer and play the file 
to listen to the recorded reply. In another preferred embodiment 
the audio file is posted on an Internet web site in a streaming 
audio format such as REALAUDIO R, and a reply e-mail notification 
is sent to the original sender having an Internet link to the 
audio file therewithin. When the original sender receives the 
reply e-mail, he can click on the link to play the audio file. 
Streaming audio formats enable the user to play an audio file on- 
line, while it is being delivered, rather than have to wait until 
the entire audio file is downloaded. REALAUDIO is a registered 
trademark of Progressive Networks, Inc. Information about 
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REALAUDIO files is available on the Internet at http://www.rea- 
laudio. com. 

Advantages of the latter embodiment are (1) the e-mail 
reply is small in size, since it only contains a link to the 
audio file, rather than the file itself; and (2) the audio file 
is streamed for on-line playing. Advantages of the former embodi- 
ment are (1) the reply e-mail is self-contained, and the user 
does not have to use a web browser to listen to the voice reply; 
and (2) sensitive information is not being broadcast over the 
web. It should be pointed out that from an ergonometric point of 
view, both embodiments have the advantage that e-mail messages 
are being replied to with speech, and the replies are being 
listened to rather than read. 

Dictating e-mail messages into a voice response unit over 
a telephone line is faster and simpler than typing a text mes- 
sage. For users who receive large volumes of e-mail on a daily 
basis, reading through e-mail is both time-consuming, and diffi- 
cult on one's eyes. Many professionals spend hours daily reading 
and replying to e-mail messages. By listening to e-mail, rather 
than reading it, these professionals can work on other "vision- 
intensive" tasks while they are listening to their e-mail mes- 
sages at the same time. This is analogous to working on a task 
while listening in on a conference call being played on the 
speaker of a telephone. 

To assist in understanding the present invention, two 
simplified example sessions are described hereinbelow: 

EXAMPLE SESSION I 
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A mobile user who wishes to send a message to an anoth- 
er person whose name appears on a predefined list of contacts, 
such as a company directory, dials up to a voice response unit 
(VRU) . The VRU responds 1. EMAIL 2. PERSONAL DIRECTORY. 3. COMPA- 
NY DIRECTORY. The user selects 3 and the VRU responds PLEASE 
ENTER THE FIRST THREE LETTERS OF THE CONTACT'S NAME . The user 
enters via the telephone touch pad the first three letters. The 
VRU responds TO SEND AN EMAIL TO <CONTACT> PLEASE PRESS 1. The 
User enters 1. The VRU responds PLEASE RECORD YOUR VOICE AT THE 
TONE, PRESS # WHEN YOU ARE FINISHED. The User records his voice 
and presses # to terminate the recording session. 

The VRU then reads a second menu of options as follows: 

1. ATTACH SPEECH REPLY AS A WAV FILE ATTACHMENT TO E-MAIL REPLY 

2. POST SPEECH REPLY AS A REALAUDIO FILE AND A WAV FILE AND SEND 
NOTIFICATION WITH LINKS Selecting option 1 instructs the VRU to 
attach the WAV file to an e-mail reply. Selecting option 2 in- 
structs the VRU to (1) convert the WAV file to a REALAUDIO file, 
(2) to post both a WAV file and a REALAUDIO file on an Internet 
web site, and (3) to send a notification reply e-mail containing 
two links — one link to the WAV file and one link to the REALAU- 
DIO file. 

EXAMPLE SESSION II 

A mobile user dials in to a voice response unit (VRU) 
by means of a telephone to access his e-mail remotely. The VRU 
reads out a message indicating that the user has e-mail in his 
inbox. Each e-mail item contains a unique identifier, data fields 
including FROM, RE and DATE, a text body, and one or more attach- 
ments. Each e-mail message is converted from text to speech by a 



text-to-speech converter within the VRU. The VRU reads out a 
first menu of options as follows: 

1. ID:018, FROM: ANDY, RE : MEETING TOMORROW, DATE : mm/dd/yy 
2. ID: 004, FROM: BILL, RE: SUNDAY PICNIC, DATE : mm/dd/yy 3. ID: 103, 
FROM: CHARLIE, RE: UPCOMING EXAM, DATE : mm/dd/yy 4. ID: 075, 
FROM: DAN, RE: DOCTOR'S APPOINTMENT, DATE: mm/dd/yy 5. <MORE> Option 
5, ,, <MORE> M , is used for menus that entail more information than 
is presented in a single list. By selecting "<MORE>" , the user 
requests additional lists of menu items. Correspondingly, succes- 
sive lists could have a "<BACK>" option for returning to previous 
lists. 

After listening to a specific e-mail message, the user 
can reply to the message and/or route the message to his fax 
machine or to his pager. For reply purposes, the user speaks into 
his telephone and the VRU records the speech as a WAV file. The 
VRU then reads a second menu of options as follows: 1. ATTACH 
SPEECH REPLY AS A WAV FILE ATTACHMENT TO E-MAIL REPLY 2 . POST 
SPEECH REPLY AS A REALAUDIO FILE AND A WAV FILE AND SEND NOTIFI- 
CATION WITH LINKS Selecting option 1 instructs the VRU to attach 
the WAV file to an e-mail reply. Selecting option 2 instructs the 
VRU to (1) convert the WAV file to a REALAUDIO file, (2) to post 
both a WAV file and a REALAUDIO file on an Internet web site, and 
(3) to send a notification reply e-mail containing two links 
one link to the WAV file and one link to the REALAUDIO file. 

In this case, the receiver has a choice of whether to 
download the WAV file to his local computer, or to stream it for 
on-line playing. In an alternate embodiment of the present inven- 
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tion, a choice between attaching a WAV file in a reply e-mail or 
posting a REALAUDIO file and a WAV file on an Internet site can 
be pre-programmed, thus avoiding the need for the second menu of 
options hereinabove. The pre-programmed choice of attach vs. post 
can be customized for each reply address. 

Similarly, in another alternate embodiment of the present 
invention, when posting his reply on an Internet web site, the 
user can be given the choice of whether to post a WAV file or a 
REALAUDIO file, or both. For routing purposes, the VRU reads a 
third menu of options as follows: 1. FAX E-MAIL TO ME 2 . FAX E- 
MAIL ATTACHMENT TO ME 3. SEND TEXT ONLY TO MY PAGER 4. <MORE> 
Selecting option 1 instructs the VRU to fax e-mail, without 
attachments, to the user's fax machine. Selecting option 2 in- 
structs the VRU to fax the e-mail attachment (s) to the user's fax 
machine. Selecting option 3 instructs the VRU to send the text 
body of the e-mail to the user's pager. 

Reference is now made to Fig. 1, which illustrates a 
voice response computer 100 used for processing voice electronic 
mail in a preferred embodiment of the present invention. Voice 
response computer 100 may include several components, as follows. 
A voice processing board 110 is used for processing incoming and 
outgoing voice messages. An example of such a board is the Dia- 
logic D/21H board for Windows, a description of which is avail- 
able at: http://www.dialogic.com. An e-mail client 120 is used 
for pulling incoming e-mail messages from local or remote e-mail 
servers, and for sending outgoing e-mail messages to local or 
remote e-mail servers. A text-to-speech converter 130 is used for 
converting text messages into voice messages. 
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Use of text-to-speech converter 13 0 is now described. 
Each e-mail message may be converted from text to speech by means 
of text-to-speech converter 130 (Fig.l). In a preferred embodi- 
ment, the present invention uses Microsoft's SAPI (Speech Appli- 
cation Programming Interface) version 4.0 toolkit for carrying 
out text-to-speech conversion. SAPI can be used with two well- 
known algorithms for text-to-speech conversion: Whistler and 
Lernout & Hauspie. Reference information for Microsoft's 
SAPI 4.0 is available on-line via the Internet at Microsoft's 
web site http://www.microsoft.com/iit/onlinedocs/intro2sapi.html, 
and reference information on Lernout & Hauspie is available on 
the Internet at http://www.lhs.com. Other text-to-speech tool- 
kits, such as the AcuVoice toolkit of Fonix Corp., are available 
and can be used instead of a Microsoft toolkit. 

Reference information on AcuVoice is available on the 
Internet at http://acuvoice.com. Text-to-speech converter 150 
(Fig. 1) stores the resultant speech data in an audio file, such 
as a WAV file. WAV is a standard file format for audio files. 
Three common types of WAV files are PCM (pulse code modulation 
audio) , which is used on compact disks (CDs) , (2) mLaw compressed 
audio, and (3) Interactive Multimedia Association ADPCM (adaptive 
differential pulse code modulation) compressed audio. 

In a preferred embodiment the present invention uses a 
raw uncompressed audio format such as PCM, with single channel 
audio sampled at 11 KHz with 8 bits per sample. Single channel, 8 
bits and 11 KHz are standard sampling parameters for speech. 

An encoder 14 0 is used to compress WAV and other audio 
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files. Use of encoder 14 0 is described more fully with reference 
to Fig. 3 below. A speech-to-text converter is used for convert- 
ing voice messages into text messages. Use of speech-to-text 
converter 150 is described more fully with reference to Fig. 5 
below. An e-mail server 160 is used as a local intermediate mail 
server for outgoing mail from voice response computer 100. 

Use of e-mail server 160 is described more fully with 
reference to Figs. 2A & 2B below. E-mail parser 170 is used for 
two purposes: (1) extracting messages and other fields from e- 
mail, and (2) breaking text into granular units such as sentences 
for managing threads that process text-to-speech tasks. Use of e- 
mail parser 170 is described more fully with reference to Fig. 7 
below. A e-mail database 180 is used for storing e-mail for users 
of a voice response system. Use of e-mail database 18 0 is de- 
scribed more fully with reference to Figs. 2A & 2B below. 

It will be appreciated by those skilled in the art upon 
reading the ensuing description that the present invention does 
not require use of all of the abovementioned components. Specifi- 
cally, the first two components, voice processing board 110 and 
e-mail client 12 0 are required. The remaining six components: 
text-to-speech converter 13 0, encoder 140, speech-to-text con- 
verter 150, e-mail server 160, e-mail parser 170 and e-mail 
database 180 are not required, but serve to provide additional 
functionality and enhancements as described hereinbelow. 

Reference is now made to Figs. 2 A and 2B, which are 
simplified illustrations of an e-mail management system that 
posts a recorded message as an audio file and posts a reply 
message as an audio file on an Internet web site, in accordance 
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with a preferred embodiment of the present invention, Fig. 2 A 
representing the functionality of posting a recorded message as 
an audio file and Fig. 2B representing the functionality of 
posting a reply message as an audio file. 

It is appreciated that Figs. 2A and 2B show essentially 
a single system that has two functionalities. It is appreciated 
that the present invention is not limited to a system having both 
functionalities but extends as well to systems having only one of 
the two functionalities. 

Reference is now made specifically to Fig. 2A, which 
illustrates an message management system that posts a recorded 
message as an audio file on an Internet web site, in accordance 
with a preferred embodiment of the present invention. 

The user, employing a telephone 200, dials in to the 
voice response computer 100 (Fig. 1) and selects a contact from 
his directory listing 202. The user then records a message into 
the voice response computer 100. The user then informs voice 
response computer 100, by means of the telephone pad, that he 
wishes to convert the recorded message into a streaming audio 
format, such as a REALAUDIO format, and to post it on an Internet 
web server 2 30. Internet web server 23 0 preferably contains an 
HTTP server 240, an e-mail server 250 and a REALAUDIO server 2 60. 

In response to the user's instruction, voice response 
computer 100 sends an outgoing e-mail containing a WAV file 
attachment, to e-mail server 250, using the SMTP protocol. Upon 
receiving the e-mail containing the WAV file attachment, Inter- 
net web server 2 30 extracts the WAV file. If the user instructed 
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voice response computer 100 to post WAV and REALAUDIO files , then 
Internet web server 230 converts the extracted WAV file into a 
REALAUDIO file, by means of REALAUDIO encoder 270 contained 
therewithin, and posts both files (the WAV file and the REALAUDIO 
file) on an Internet web site. 

Internet web server 230 then sends an outgoing notifi- 
cation, typically in the form of an e-mail, to the contact, via 
the contact's e-mail server 280, using the SMTP protocol. As an 
alternative to an e-mail notification, the present invention may 
employ an alternative type of notification, such as a notifica- 
tion which employs POTs telephone voice messaging, a beeper, an 
AOL Interactive Messaging (AIM) client or an ICQ client. For the 
sake of simplicity and conciseness, reference is made hereinbelow 
only to e-mail notifications, notwithstanding that the invention 
is not limited thereto. 

The notification e-mail contains links to the REALAUDIO 
file and to the WAV file. Internet web server 23 0 is better 
suited for sending the notification e-mail than is voice response 
computer 100, since the Internet web server knows precisely when 
the REALAUDIO file and the WAV are posted and available for 
remote access . 

In order to send the outgoing e-mail containing the WAV 
file attachment from voice response computer 100 to e-mail server 
250, it is only necessary for voice response computer 100 to use 
e-mail client 120 (Fig. 1) . However, in a preferred embodiment of 
the present invention, voice response computer 100 also contains 
its own local e-mail server 160 (Fig. 1) . The outgoing e-mail 
containing the WAV file attachment is sent by voice response 
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computer 100 to local e-mail server 160. E-mail served 160 then 
sends the e-mail containing the WAV file to e-mail server 250. 

. The advantage of using a local e-mail server within voice 
response computer 100 is the time saving in waiting for an ac- 
knowledgment from the e-mail server receiving the e-mail. When an 
e-mail client sends e-mail, it must wait for an acknowledgment 
from an e-mail server. When e-mail contains a large attachment , 
such as a WAV file, there is a significant delay in waiting for 
the acknowledgment, due to bandwidth limitations of transmission 
over the Internet. 

The system of Fig. 2A regularly sends e-mail messages 
with large WAV file attachments, and were the e-mail client 
required to wait for an acknowledgment each time a reply e-mail 
is sent, the latency of the system would be prolonged. When an e- 
mail message is sent to a local e-mail server, an acknowledgment 
is received almost immediately. Thus by sending the e-mail con- 
taining the WAV file attachment to its own local e-mail server 
160 (Fig. 1) , voice response computer 100 achieves better 
throughput and performance. 

From the contact's e-mail server 280, the notification e- 
mail is retrieved by an e-mail client on the contacts 1 s local 
computer 290, as an incoming e-mail, typically using the POP 
protocol. As described hereinbelow, the e-mail client resident in 
the contact's computer 290 may be an audio e-mail client 300 of 
the present invention. Upon receiving the notification e-mail, 
the contact clicks on a link contained in the notification e-mail 
to access the WAV file or the REALAUDIO file, using the HTTP 



protocol . 

The contact's computer 290 contains a REALAUDIO player 
310 for playing REALAUDIO files, and a WAV player 320 for playing 
WAV files. It may be the case that the contact is also a sub- 
scriber to the system of the present invention, as is the user. 
In this case the contact can also call in to server 280, by means 
of telephone 330, to retrieve his e-mail. 

For simplicity, it is assumed in Fig. 1 that the con- 
tact's e-mail server 280 also functions as a voice response 
computer similar to voice response computer 100. Otherwise, the 
contact calls in to a voice response computer (not shown) that 
pulls the contact's e-mail from e-mail server 280. In a preferred 
embodiment of the present invention, the contact's voice response 
computer (not shown) has the capability to automatically download 
audio files linked to a reply e-mail, and to play the audio files 
to the contact. 

In this embodiment, when a notification e-mail with a 
link to an audio WAV file arrives at the contact's e-mail server 
280, and is thereafter received by the contact's voice response 
computer, the contact's voice response computer automatically 
downloads the WAV file linked to the reply e-mail and plays it to 
the contact when he dials in. 

It can thus be appreciated that the contact who is also 
a subscriber to the system of the present invention does not need 
to download the WAV file from the Internet himself, and can 
listen to the contents of the WAV file simply by dialing in to 
his voice response computer. In a preferred embodiment of the 
present invention, voice response computer 100 contains its own 



e-mail database 180 (Fig, 1) for storing the user's e-mail. E- 
mail database 180 provides a structured directory-type facility 
for storing voice electronic mail. The user can save and organize 
his voice electronic mail into individual "mailboxes," for future 
access. 

Reference is now made to Fig. 2B, which illustrates an 
e-mail management system that posts a reply message as an audio 
file on an Internet web site, in accordance with a preferred 
embodiment of the present invention. Elements in Fig. 2B, which 
are identical to those in Fig. 2A are identified by the same 
reference numerals . 

In the discussion of Figs. 2B - 7 that follows, it is 
being assumed that a "user" receives an e-mail message from a 
"sender," and generates a speech reply. The term "user" is indic- 
ative of the fact that the user under discussion is a subscriber 
to the system of the present invention. 

There exist two alternative methods for providing e- 
mail to e-mail database 180, namely a pull method and a push 
method. These two methods may operate concurrently. 

According to a first method, which is a pull method, e- 
mail client 120 (Fig. 1) polls e-mail server 210 on a regular 
basis (for example, every ten minutes) to determine the contents 
of the user's current inbox of e-mail, and to synchronize the e- 
mail items in e-mail database 180 of voice response computer 100 
with the e-mail items in server 210. 

Specifically, voice response computer 100 pulls newly 
arrived e-mail that has not yet been downloaded from e-mail 



server 210 and purges old e-mail that is not longer present in e- 
mail server 210, and that has not been stored by the user in a 
mailbox for archiving* As described hereinbelow, e-mail database 
18 0 (Fig. 1) may be used to store and control the e-mail items in 
voice response computer 100. 

According to a second method, which is a push method, 
an e-mail server 215, upon receiving an e-mail for a user, in- 
serts the e-mail directly into e-mail database 180. 

In a preferred embodiment of the present invention, the 
user dials in to voice response computer 100 from telephone 220 
to retrieve his e-mail from e-mail database 180. By pressing on 
the telephone pad in accordance with menu options read from the 
VRU, as illustrated in Example Session II, the user selects e- 
mail messages to be read to him by voice response computer 100. 

After listening to an e-mail message, the user dictates 
a reply into the telephone, and voice response computer 100 
records the reply. The user then informs voice response computer 
100, by means of the telephone pad, that he wishes to convert the 
recorded reply into a streaming audio format, such as a REALAUDIO 
format, and to post it on an Internet web server 23 0. Internet 
web server 230 contains an HTTP server 240, an e-mail server 250 
and a REALAUDIO server 260. As described hereinbelow, e-mail 
server 210 and e-mail server 250 may be the same e-mail server. 

In response to the user's instruction, voice response 
computer 100 sends an outgoing reply e-mail containing a WAV file 
attachment to e-mail server 250, using the SMTP protocol. Upon 
receiving the reply e-mail containing the WAV file attachment, 
Internet web server 2 30 extracts the WAV file. If the user in- 
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structed voice response computer 100 to post WAV and REALAUDIO 
files, then Internet web server 23 0 converts the extracted WAV 
file into a REALAUDIO file, by means of REALAUDIO encoder 270 
contained therewithin, and posts both files (the WAV file and the 
REALAUDIO file) on an Internet web site. 

Internet web server 230 then sends an outgoing notifica- 
tion e-mail to the sender, via a sender's e-mail server 280, 
using the SMTP protocol. The notification e-mail contains links 
to the REALAUDIO file and to the WAV file. Internet web server 
230 is better suited for sending the notification e-mail than is 
voice response computer 100 or e-mail server 210, since Internet 
web server knows precisely when the REALAUDIO file and the WAV 
are posted and available for remote access. 

In order to send the outgoing reply e-mail containing the 
WAV file attachment from voice response computer 100 to e-mail 
server 250, it is only necessary for voice response computer 100 
to use e-mail client 120 (Fig. 1) . However, in a preferred embod- 
iment of the present invention, voice response computer 100 also 
contains its own local e-mail server 160 (Fig. 1) . The outgoing 
reply e-mail containing the WAV file attachment is sent by voice 
response computer 100 to local e-mail server 160. E-mail server 
160 then sends the reply e-mail containing the WAV file to e-mail 
server 2 50. 

The advantage of using a local e-mail server within voice 
response computer 100 is the time saving in waiting for an ac- 
knowledgment from the e-mail server receiving the e-mail. When an 
e-mail client sends e-mail, it must wait for an acknowledgment 




from an e-mail server. When e-mail contains a large attachment, 
such as a WAV file, there is a significant delay in waiting for 
the acknowledgment, due to bandwidth limitations of transmission 
over the Internet, 

The system illustrated in Fig. 2B regularly sends e-mail 
replies with large WAV file attachments, and were the e-mail 
client required to wait for an acknowledgment each time a reply 
e-mail is sent, the latency of the system would be prolonged. 
When an e-mail reply is sent to a local e-mail server, an ac- 
knowledgment is received almost immediately. Thus by sending the 
reply e-mail containing the WAV file attachment to its own local 
e-mail server 160 (Fig. 1) , voice response computer 100 achieves 
better throughput and performance. 

From the sender's e-mail server 28 0, the notification e- 
mail is retrieved by an e-mail client on the sender 1 s local 
computer 290, as an incoming e-mail, using the POP protocol. As 
described hereinbelow, the e-mail client resident in the sender's 
computer 290 may be an audio e-mail client 300 of the present 
invention. Upon receiving the notification e-mail, the sender 
clicks on a link contained in the notification e-mail to access 
the WAV file or the REALAUDIO file, using the HTTP protocol. 

The sender's computer 290 contains a REALAUDIO player 310 
for playing REALAUDIO files, and a WAV player 320 for playing WAV 
files. It may be the case that the sender is also a subscriber to 
the system of the present invention, as is the user. In this case 
the sender can also call in to server 280, by means of telephone 
330, to retrieve his e-mail. 

For simplicity, it is assumed in Fig. 1 that the sender's 
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e-mail server 280 also functions as a voice response computer 
similar to voice response computer 100. Otherwise, the sender 
calls in to a voice response computer (not shown) that pulls the 
sender 1 s e-mail from e-mail server 280. In a preferred embodiment 
of the present invention, the sender's voice response computer 
(not shown) has the capability to automatically download audio 
files linked to a reply e-mail, and to play the audio files to 
the sender. 

In this embodiment, when a notification e-mail with a 
link to an audio WAV file arrives at the sender 1 s e-mail server 
280, and is thereafter received by the sender" s voice response 
computer, the sender's voice response computer automatically 
downloads the WAV file linked to the reply e-mail and plays it to 
the sender when he dials in. 

It can thus be appreciated that the sender who is also a 
subscriber to the system of the present invention does not need 
to download the WAV file from the Internet himself, and can 
listen to the contents of the WAV file simply by dialing in to 
his voice response computer. In a preferred embodiment of the 
present invention, voice response computer 100 contains its own 
e-mail database 180 (Fig. 1) for storing the user's e-mail. E— 
mail database 180 provides a structured directory-type facility 
for storing voice electronic mail. The user can save and organize 
his voice electronic mail into individual "mailboxes," for future 
access . 

E-mail database 180 is updated whenever e-mail client 120 
(Fig. l) polls e-mail server 210 for the user's e-mail. When new 
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e-mail has arrived at e-mail server 210, e-mail client 12 0 down- 
loads it to voice response computer 100, and adds it to database 
180. When old e-mail is removed from e-mail server 210, voice 
response computer 100 removes its copy of such old e-mail from 
database 180, if the old e-mail has not been stored by the user 
in a mailbox. In this way, database 180 maintains synchronization 
with e-mail server 210. Database 180 can be an SQL database with 
fields such as sender, subject and date. Voice response computer 
100 can carry out user requests for searches within database 180. 
For example, a user may want to access the e-mail that was sent 
from a designated sender within a designated time period. 

Reference is now made to Fig. 3, which illustrates an 
e-mail management system that sends an audio file attachment to 
an e-mail, such as a reply e-mail, in accordance with a preferred 
embodiment of the present invention. As above, the user dials in 
to voice response computer 100 from telephone 220 and dictates an 
e-mail to a recipient. The e-mail may or may not be a reply e- 
mail and the recipient may or may not be the sender of an e-mail 
to which a reply is being sent. 

In the embodiment depicted in Fig. 3, the user informs 
voice response computer 100 that he desires to convert his re- 
corded message, such as a reply, into an audio file, such as a 
WAV file, and to send it to an e-mail server 280 of a recipi- 
ent, such as the sender, as an attachment to an e-mail, such as a 
reply e-mail. As described hereinabove with reference to Figs. 2A 
& 2B, in a preferred embodiment of the present invention voice 
response system 100 sends the e-mail containing the WAV file 
attachment to its own local e-mail server 160 (Fig. 1) , and then 



e-mail server 160 sends it to e-mail server 280. This reduces 
network latency caused by the delay in waiting for an acknowledg- 
ment from a remote e-mail server. 

The recipients computer 290 pulls the e-mail as an 
incoming message from the recipient's e-mail server 280, typical- 
ly using the POP protocol. By clicking on the reference to the 
WAV file attachment in the e-mail, the recipient can either save 
the file to his local disk, or play it using WAV player 320, or 
both. In an alternative embodiment of the present invention the 
audio file attached to a e-mail can be compressed as a self- 
extracting compressed file. Self -extracting files contain decom- 
pressors therewithin . 

The recipient of an e-mail message with a self-extracting 
file attached therewithin can click on the reference to the file, 
and the compressed audio file automatically decompresses itself. 
For example, the present invention can attach a zipped WAV file 
to an e-mail. The attachment automatically unzips itself and re- 
generates the WAV file when a recipient clicks on the reference 
to the attachment. It is for the purpose of this alternative 
embodiment that voice response computer 100 contains encoder 14 0 
(Fig. l) for compressing audio files. 

Reference is now made to Fig. 4, which illustrates how a 
user can route his e-mail to a pager and/or a fax machine, in 
accordance with a preferred embodiment of the present invention. 
The user can instruct voice response computer 100 to route his 
incoming e-mail message by sending it as a text message to his 
pager 410, or to his fax machine 420, or to both pager 410 and 
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fax machine 4 20. 

If the user elects to route his e-mail to pager 410, 
voice response computer 100 typically sends an outgoing e-mail 
message to e-mail gateway 430 , using the SMTP protocol. Gateway 
4 30 operates by accepting e-mail as input and producing a message 
that can be delivered to a pager as output. Specifically, the 
Simple Messaging System (SMS) protocol is designed as a protocol 
for transferring text data to a thin wireless data terminal, such 
as a pager. E-mail gateway 430 converts the user's text e-mail 
into a form suitable for wireless transmission to pager 410. 
Alternatively, voice response computer 100 can communicate in the 
SMS protocol directly with pager 410. 

If the user elects to route his e-mail to fax machine 
420, voice response computer 100 typically sends an outgoing e- 
mail message to e-mail gateway 440, using the SMTP protocol. 
Gateway 440 operates by accepting e-mail as input and producing a 
message that can be transmitted to a fax machine as output. As 
such, e-mail gateway 440 converts the user's text e-mail into a 
form suitable for transmission to fax machine 420. Alternatively, 
voice response computer 100 can communicate directly with fax 
machine 42 0. 

Reference is now made to Fig. 5, which illustrates how a 
user can send an e-mail, such as a reply e-mail, to a pager or a 
fax machine, in accordance with a preferred embodiment of the 
present invention. As described hereinabove with reference to 
Figs. 2A, 2B and 3, the present invention enables the user to 
send an e-mail, such as an e-mail reply, by dictating into a 
telephone. The speech message is recorded, stored as an audio 




file and delivered to the recipient. 

Fig. 5 illustrates the alternative of converting the 
speech message into a text message, and sending the text message 
to the recipients pager 510, or to the recipients fax machine 
52 0, or to both. It is for the purpose of this feature that voice 
response computer 100 contains speech-to-text converter 150 (Fig. 
1) for producing a text message from speech dictated into the 
user's telephone 220 (Fig. 2). Voice response computer 100 con- 
verts the user's speech reply to a text message, by means of 
speech-to-text converter 150. If the user elects to deliver the 
text message to pager 510, voice response computer 100 typically 
sends the text message as an outgoing e-mail to e-mail gateway 
530, using the SMTP protocol. 

If the user elects to deliver the text message to fax 
machine 52 0, voice response computer typically sends the text 
message as an outgoing e-mail to e-mail gateway 540, using the 
SMTP protocol. Although it is the "un-wired" mobile user who 
typically accesses his e-mail through a telephone unit, the 
present invention is also advantageous to the "wired" user who 
accesses his e-mail through a computer connected to a network. 

For reasons of convenience, and other reasons as well, 
a user with a computer may nevertheless prefer to reply to e-mail 
messages and send new e-mail messages using speech rather than 
text. An indicator of this is the emergence of IP phones for 
speech communication over the Internet. 

The present invention also provides an audio e-mail 
client that manages incoming and outgoing audio messages. It 
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enables a user to dictate a voice e-mail message by speaking into 
a microphone connected to his computer, and to convert the speech 
into an audio file. The audio file can then be either (1) sent as 
an attachment to a standard e-mail message, or (2) posted on an 
Internet web site. In the latter case, a notification e-mail 
message with a link to the audio file is sent. It can thus be 
appreciated that the present invention provides voice mail-like 
capability to a computer user. 

The audio e-mail client of the present invention may 
enable a user to save an audio file that is in a streaming audio 
format on a remote server computer or on his local computer. This 
feature is not available in standard REALAUDIO players. Such 
players are used for listening to live broadcasts over the Inter- 
net, such as newscasts, and are typically used to play an audio 
file once. The present invention introduces a novel use of 
streaming audio files; namely, for voice e-mail messages, and in 
this context it is important to be able to save the voice mes- 
sages for future reference. In a preferred embodiment, the audio 
e-mail client of the present invention provides a mail archive 
for voice messages that is structured hierarchically in files and 
directories, as is commonly done today for text e-mail messages 
using familiar e-mail clients such as EUDORA, MICROSOFT INTERNET 
EXPLORER and NETSCAPE COMMUNICATOR. 

When an incoming voice e-mail message is saved, the user 
can direct it to a specific directory or sub-directory, for 
cataloging and indexing purposes. Moreover the incoming voice e- 
mail message can be saved in a variety of formats, such as REA- 
LAUDIO, WAV or compressed WAV. 
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Referring back to Fig. 2B, in a preferred embodiment of 
the present invention the sender's computer 290 contains such an 
audio e-mail client 3 00, to play the REALAUDIO file posted by 
Internet web server 2 30 and to save the file on Internet web 
server 2 30 or on the sender's computer 290. A similar structure 
is applicable to a recipient using the functionality of Fig. 2A. 
In a preferred embodiment, the audio e-mail client of the present 
invention also carries out remote functions by posting voice e- 
mail messages on an Internet web server. 

Specifically, a user can post voice e-mail messages on an 
Internet web site for retrieval by another person, or for pur- 
poses of archiving his own voice e-mail. Audio e-mail client 300 
has the capability to manage RAM, hard diskspace and files on 
remote server computers, by using an appropriate protocol for 
uploading and storing REALAUDIO and other audio files on Internet 
web server 23 0. This is indicated in Fig. 2B by two-directional 
arrows between Internet web server 230 and audio e-mail client 
300. Similarly, audio e-mail client 300 is used in the embodiment 
illustrated by Fig. 3 for the purpose of file management on 
remote server computers. 

Specifically, audio e-mail client 300 can archive and 
manage the WAV files it receives by storing them on a remote 
server computer such as Internet web server 230 (Figs. 2A & 2B) . 
The sender's voice electronic mail is thus stored on a remote 
computer, and accessible from his local computer. This is similar 
to the remote mail storage that Hotmail provides to its subscrib- 
ers for text electronic mail. Reference information on Hotmail is 
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available at http://www.hotmail.com. 

In a preferred embodiment of the present invention, 
Internet web server 230 is part of an overall electronic voice 
mail management system. The overall management system also in- 
cludes voice response computer 100 and, optionally, e-mail server 
210. The protocol for uploading and storing files on Internet web 
server 230 may be proprietary, and, if so, audio e-mail client 
300 sends files and instructions to Internet web server 230 in 
accordance with such proprietary protocol . 

It should be apparent to those skilled in the art that 
various processing units illustrated in the various figures may 
reside in separate computers, or in the same computer. For exam- 
ple, referring to Figs. 2A & 2B, voice response computer 100 and 
e-mail server 210 and Internet web server 23 0 may all reside 
within the same computer. Referring to Fig. 1, e-mail client 120 
may reside in a different computer than voice response computer 
100, and, as such, need not be contained within voice response 
computer 100. 

The system of the present invention is a multi-user 
system. As such, it should also be apparent to those skilled in 
the art that various processing units depicted in the figures may 
be present multiple times within the system of the present inven- 
tion. For example, referring to Fig. 1, voice response computer 
100 may contain several e-mail clients 120. 

Reference is now made to Fig. 6, which illustrates two 
users dialing in to a voice response computer operative in ac- 
cordance with a preferred embodiment of the present invention. 
The two users are referred to as "user #1" and "user #2." Voice 
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response computer 100 pulls e-mail from e-mail server 210 con- 
taining user #l's e-mail, and from a second e-mail server 610 
containing user #2*s e-mail. User #1 dials in to voice response 
computer 100 by means of telephone 220, and user #2 dials in by 
means of telephone 62 0. 

Voice response computer 100 is connected to two e-mail 
clients 63 0 that work in parallel. Were a single e-mail client to 
be used, there would be only one client pulling e-mail for user 
#1 and user #2. Voice response computer 100 may also contain 
multiple text-to-speech converters 13 0 (Fig. 1) and multiple e- 
mail parsers 170 (Fig. 1) , for simultaneously handling multiple 
users, rather than processing the user messaging tasks in a 
single queue. For multi-user systems such as the systems of the 
present invention, the management of processing threads is a 
significant factor in overall system performance. 

One of the recurrent processing tasks performed by the 
system of the present invention is the conversion of e-mail 
messages from text to speech by text-to-speech converter 13 0 
(Fig. 1) . Voice response computer 100 routinely processes an 
enormous volume of e-mail messages. Were a separate processing 
thread to be allocated for each text-to-speech task, the prolif- 
eration of threads could choke the system, resulting in bad 
performance. In a preferred embodiment, the present invention 
uses a small number of threads and uses a waiting queue for text- 
to-speech tasks. 

Reference is now made to Fig. 7, which is a simplified 
illustration of a queue for text-to-speech processing tasks in a 
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preferred embodiment of the present invention • For simplicity a 
single thread 710 is illustrated. Thread 710 handles each 
text-to-speech task, and carries out a function described as 
DO(t, f ) , where t is a text string and f is an audio file. The 
function DO converts text string t to speech, and saves the 
result in audio file f, using text-to-speech converter 130. 

Shown in Fig. 7 are two e-mail messages 720 and 73 0 
that have been downloaded to voice response computer 100 by e- 
mail client 120 (Fig. 1) . Each e-mail message is to be processed 
by thread 710, for conversion from text to speech. The processing 
requests for messages 72 0 and 730 are temporarily stored in a 
waiting queue 740. As thread 710 completes each task, a new task 
is delivered by waiting queue 74 0. 

One problem encountered by such a thread management 
system is that of "blocking." Blocking occurs when a task that 
requires a large amount of processing blocks other tasks that 
require little processing. For example, suppose message 720 is a 
large message and message 730 is a short message, and that mes- 
sage 720 is ahead of message 730 in queue 740. Then the text-to- 
speech processing for message 730 would be deferred until message 
720 is completely converted into speech. 

The present invention overcomes the problem of blocking, 
in a preferred embodiment, by parsing each message into small 
units, such as individual sentences, by e-mail parser 170. Each 
sentence is then individually sent to queue 740 for input to 
thread 710 as a separate text-to-speech conversion task. In this 
way, the short message 730 may enter queue 740 at a queue posi- 
tion between the sentences of message 720, and thus be processed 
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by thread 710 before the processing of message 720 is finished. 
E-mail parser 170 may use punctuation marks as a basis for pars- 
ing messages into individual sentences. 

E-mail parser 17 0 may include a pre-parsing module 
which, in a preferred embodiment of the present invention is 
implemented as a PERL module. The pre-parsing module may perform 
text substitutions, insertions and deletions which may improve 
the understandability of text to speech conversions. 

It should be apparent to those skilled in the art that 
other methods of parsing may be used instead, and that messages 
may be parsed into units other than sentences as well. For exam- 
ple, for uniformity of size, messages may be parsed into units of 
consecutive words whose total size does not exceed 100 charac- 
ters. It should also be apparent to those skilled in the art that 
multiple e-mail parsers 170 may be present within voice response 
computer 100, in order to process multiple e-mail messages in 
parallel. 

The present invention is also useful within a "buddy 
system." Such a system is based upon a "buddy list," which is a 
list of friends, or members, who wish to be notified when any 
members of the list are on-line. An example of such a system is 
the Mirabilis ICQ ("I seek you") system for notifying the other 
members when a member of the list is connected to the Internet. 
Reference information on the ICQ system is available on the 
Internet at http : //www . icq . com . 

Within the context of the present invention, voice re- 
sponse computer 100 can be programmed to notify members of a 
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"buddy list" whenever a user who is one of the members dials in 
to his voice response computer. Upon notification the members are 
alerted that the user is currently dialed in to his voice re- 
sponse computer, and, as such, can be contacted therefrom. It 
should be appreciated that the buddy list notification may be 
employed by the user to send a message to a buddy. 

Reference is now made to Fig. 8 which is a simplified 
illustration of initiation of a voice e-mail message from a 
telephone in accordance with a preferred embodiment of the 
present invention. As seen in Fig. 8, a person who wishes to send 
a voice e-mail uses a POTS telephone to call a voice response 
unit (VRU) . Once a connection has been made to the VRU, the 
sender may enter an intended recipient's e-mail address either by 
choosing an entry from the pre-defined directory listing, via 
DTMF codes on the POTS telephone or alternatively or additionally 
by speaking via a speech recognition module. 

The sender is typically prompted to record a voice mes- 
sage and does so. The VRU then generates a unique identifier 
(UID) for the voice message and transmits the recorded voice 
message, typically in a WAV format to an event controller. The 
event controller preferably saves the WAV formatted data to a 
file converts it to a REALAUDIO file and sends an e-mail to the 
intended recipient with a link to the REALAUDIO and the WAV 
files. The event controller also preferably inserts an audio file 
listing database information corresponding to the voice message, 
typically including the UID, the particulars of the sender, the 
subject of the voice message and the links to the REALAUDIO and 
WAV files. 
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The recipient receives the e-mail , which provides the 
link with the UID, and clicks on the link to receive the voice 
message. The recipient's computer contacts the web server, which 
connects to the audio file listing database and retrieves a link 
to the voice message. The link is employed by the recipient's 
computer to play the voice message. 

Reference is now made to Fig. 9, which is a simplified 
illustration of initiation of a voice e-mail message using both a 
web server and a telephone. In the embodiment of Fig. 9, a person 
intending to send a voice message submits a form bearing the 
intended recipient's e-mail address as well as his own e-mail 
address and the subject of the voice message. The form is direct- 
ed to a web server which generates a UID and transmits the UID to 
the sender. The sender then telephones a VRU using a POTS tele- 
phone and enters the UID. The sender is typically prompted to 
record a voice message and does so. The VRU then transmits the 
recorded voice message, typically in a WAV format to an event 
controller. 

The event controller preferably saves the WAV formatted 
data to a file, converts it to a REALAUDIO file and sends an e- 
mail to the intended recipient with a link to the REALAUDIO and 
the WAV files. The event controller also preferably inserts an 
audio file listing database information corresponding to the 
voice message, typically including the UID, the particulars of 
the sender, the subject of the voice message and the links to the 
REALAUDIO and WAV files. 

The recipient receives the e-mail which provides the link 
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with the UID and clicks on the link to receive the voice message. 
The recipients computer contacts a web server which connects to 
the audio file listing database and retrieves a link to the voice 
message, which link is employed by the recipient's computer to 
play the voice message* 

Reference is now made to Fig. 10, which is a simplified 
illustration of initiation of a voice e-mail message using a 
telephone having a web interface. In the embodiment of Fig. 10, a 
person intending to send a voice message using a telephone web 
interface, such as an HDML UPLink enabled hand-held device com- 
mercially available from suppliers such as Motorola (see 
http://www.phone.com), submits a form bearing the intended recip- 
ient's e-mail address as well as his own e-mail address and the 
subject of the voice message. 

The form is directed to a web server which generates a 
UID and an HDML page and transmits the UID hidden in the HDML 
page to the sender. The sender then initiates a telephone call to 
a VRU, typically by pressing a button on the UPLink device. The 
sender is typically prompted to record a voice message and does 
so. The VRU then transmits the recorded voice message, typically 
in a WAV format to an event controller. 

The event controller preferably saves the WAV formatted 
data to a file, converts it to a REALAUDIO file and sends an e- 
mail to the intended recipient with a link to the REALAUDIO and 
the WAV files. The event controller also preferably inserts an 
audio file listing database information corresponding to the 
voice message, typically including the UID, the particulars of 
the sender, the subject of the voice message and the links to the 
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REALAUDIO and WAV files. 

The recipient receives the e-mail which provides the link 
with the UID and clicks on the link to receive the voice message. 
The recipients computer contacts a web server which connects to 
the audio file listing database and retrieves a link to the voice 
message, which link is employed by the recipient's computer to 
play the voice message. 

Reference is now made to Fig. 11, which is a simplified 
illustration of a method of sending a message, such as an e-mail 
message, using a telephone having a web interface. A person 
intending to send a message, with or without a voice portion, 
enters an intended recipient's e-mail address using any suitable 
method, such as those described hereinabove with reference to any 
of Figs. 8, 9 and 10. Another method of choosing a recipient is 
to use an HDML enabled telephone and to choose a buddy contact 
from a buddy list. 

The sender determines the contents of the message, 
which may or may not include voice, by employing a VRU and in- 
structing the VRU to send the message or alternatively by access- 
ing an HDML page on a web server. 

When a VRU is employed, the VRU inserts into a message 
content database the particulars of the contents as well as the 
UID and the particulars of the sender. If a recorded voice is 
part of the message, the remainder of the procedure may be simi- 
lar to that described hereinabove with respect to any of Figs. 8 
- 10. Otherwise, the VRU may send the e-mail message directly to 
the recipient. 
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Alternatively , the web server may send a message to the 
recipient's buddy client using a buddy protocol. 

It will be appreciated by persons skilled in the art that 
the present invention is not limited by what has been particular- 
ly shown and described hereinabove. Rather the present invention 
includes combinations and sub-combinations of the various fea- 
tures described hereinabove as well as modifications and exten- 
sions thereof which would occur to a person skilled in the art 
and which do not fall within the prior art. 



41 



1 

CLAIMS 

1. A voice communication system comprising: 

a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of telephones via said telephone network for commu- 
nicating voice received via said one of said multiplicity of 
telephones via e-mail over said computer network. 

2. A voice communication system comprising: 

a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of voice response computers via said computer net- 
work for receiving voice communicated via e-mail over said 
computer network and providing a voice output to a telephone via 
said telephone network. 

3. A voice communication system comprising: 

a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 
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a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of telephones via said telephone network for commu- 
nicating voice received via said one of said multiplicity of 
telephones via e-mail over said computer network, each voice 
response computer also being actuable by an input received from 
one of said multiplicity of voice response computers via said 
computer network for receiving voice communicated via e-mail 
over said computer network and providing a voice output to a 
telephone via said telephone network. 

4. A voice communication system comprising: 

a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of telephones via said telephone network for commu- 
nicating voice received via said one of said multiplicity of 
telephones via a non-streaming Internet protocol over said com- 
puter network. 

5. A voice communication system comprising: 
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a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 

a computer network having a multiplicity of nodes and 
enabling non-streaming Internet protocol communication between 
said nodes ; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of voice response computers via said computer net- 
work for receiving voice communicated via e-mail over said 
computer network and providing a voice output to a telephone via 
said telephone network. 

6. A voice communication system comprising: 

a telephone network including a multiplicity of tele- 
phones interconnected by telephone network interconnections; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of voice response computers, each voice 
response computer being connected to a node of said computer 
network and being actuable by an input received from one of said 
multiplicity of telephones via said telephone network for commu- 
nicating voice received via said one of said multiplicity of 
telephones via a non-streaming Internet protocol over said com- 
puter network, each voice response computer also being actuable 
by an input received from one of said multiplicity of voice 
response computers via said computer network for receiving voice 
communicated via a non-streaming Internet protocol over said 
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computer network and providing a voice output to a telephone via 
said telephone network. 

7 . A voice communication system according to any of the 
preceding claims and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network. 

8 . A voice communication system according to any of the 
preceding claims and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network indicating that a user is communicating using a user's 
telephone via said telephone network with a user's voice response 
computer . 

9. A voice communication system according to any of the 
preceding claims and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network indicating that a user has communicated voice via said 
telephone network and said computer network using a user's tele- 
phone and a user's voice response computer. 

10. A voice communication system according to any of claims 
7 - 9 and wherein said voice response computer is operative to 
convert DTMF to a buddy communication protocol. 

11. A communication system comprising: 

a cellular telephone network including a multiplicity 
of telephones interconnected by telephone network interconnec- 
tions ; 



a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of computers, each computer being con- 
nected to a node of said computer network and being actuable by 
an input received from one of said multiplicity of telephones via 
said telephone network for communicating messages received via 
said one of said multiplicity of telephones via HDML over said 
computer network. 

12. A communication system comprising: 

a cellular telephone network including a multiplicity 
of telephones interconnected by telephone network interconnec- 
tions ; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 

a multiplicity of computers, each computer being 
connected to a node of said computer network and being actuable 
by an input received from one of said multiplicity of voice 
response computers via said computer network for receiving mes- 
sages communicated via HDML over said computer network and 
providing a HDML output to a telephone via said telephone net- 
work. 

13. A communication system comprising: 

a cellular telephone network including a multiplicity 
of telephones interconnected by telephone network interconnec- 
tions ; 

a computer network having a multiplicity of nodes and 
enabling e-mail communication between said nodes; 



a multiplicity of computers, each computer being con- 
nected to a node of said computer network and being actuable by 
an input received from one of said multiplicity of telephones via 
said telephone network for communicating messages received via 
said one of said multiplicity of telephones via HDML over said 
computer network, each computer also being actuable by an input 
received from one of said multiplicity of computers via said 
computer network for receiving messages communicated via e-mail 
over said computer network and providing a HDML output to a 
telephone via said telephone network. 

14. A communication system according to any of the preced- 
ing claims 11 - 13 and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network. 

15. A communication system according to any of the preced- 
ing claims 11 - 14 and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network indicating that a user is communicating using a user's 
telephone via said telephone network with a user's voice response 
computer . 

16. A communication system according to any of the preced- 
ing claims 11 - 15 and also providing buddy functionality whereby 
communications are sent to user-selected buddies via said comput- 
er network indicating that a user has communicated voice via said 
telephone network and said computer network using a user's tele- 
phone and a user's voice response computer. 
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17. A voice communication system according to any of claims 

14 - 16 and wherein said voice response computer is operative to 
convert DTMF to a buddy communication protocol. 

x^ For the Applic ant, 

Sanford T. Colb & Co. 
C: 33869 
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